Search results

Surveys in multinational, multiregional and multicultural contexts (or “3MC” surveys) are becoming increasingly important to global and regional decision-making and theory building. To serve this purpose, the surveys need to be well managed, with an awareness of key sources of survey error and how to minimize them, mechanisms in place to control the implementation process and an ability to intervene in that process when necessary in a spirit of continuous improvement (Pennell et al., 2017). One key approach for managing and assessing the quality of 3MC surveys is the total survey error (TSE) framework and associated survey process quality. This paper aims to examine the application of the TSE framework and survey process quality to the Programme for the International Assessment of Adult Competencies (PIAAC).

The authors begin with a background on TSE and discuss recent adaptations of TSE and survey process quality for 3MC surveys. They then presents a TSE framework tailored with examples of potential contributions to error for PIAAC and ways to address those through effective quality assurance (QA) and quality control (QC) approaches.

Overall, the authors find that the design and implementation of the first cycle of PIAAC largely reflect the current best practice for 3MC surveys. However, the authors identify several potential contributions to error that may threaten comparability in PIAAC and ways these could be addressed in the upcoming cycle.

With a view toward continuous improvement, the final section draws on the survey process quality approach adapted from Hansen et al.’s study (2016) to summarize the recommendations in terms of additional QA elements (inputs and activities) and associated QC elements (measures and reports) for PIAAC’s consideration in the next cycle.

At the invitation of the Programme for the International Assessment of Adult Competencies (PIAAC), this paper aims to examine advances in survey interviewer monitoring and make recommendations on minimizing the effect of interviewers on survey results.

The authors first provide an overview of the most recent literature on interviewer effects, quality assurance and quality control. Here, they draw upon recent publications such as the cross-cultural survey guidelines (www.ccsg.isr.umich.edu) and newly published or in-press material specifically addressing these issues in multicultural, multinational and multiregional (3MC) contexts.

The authors discuss trends and innovations in quality assurance and quality control in 3MC studies and draw upon examples from international surveys that are using cutting-edge and innovative approaches to monitor interviewer behavior and minimize interviewer effects.

With a view to continuous quality improvement, the authors conclude with concrete recommendations for PIAAC to consider for the next cycle. Many of the recommendations have general relevance for other large-scale cross-national surveys.